Data processing method and apparatus for convolutional neural network

ABSTRACT

A data processing method for a convolutional neural network includes: (a) obtaining a matrix parameter of an eigenmatrix; (b) reading corresponding data in an image data matrix from a first buffer space based on the matrix parameter through a first bus, to obtain a next to-be-expanded data matrix, and sending and storing the to-be-expanded data matrix to a second preset buffer space through a second bus; (c) reading the to-be-expanded data matrix, and performing data expansion on the to-be-expanded data matrix to obtain expanded data; (d) reading a preset number of pieces of unexpanded data in the image data matrix, sending and storing the unexpanded data to the second preset buffer space, and updating, based on the unexpanded data, the to-be-expanded data matrix; and (e). repeating (c) and (d) until all data in the image data matrix is completely read out on the to-be-expanded data matrix.

RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 16/250,204, filed on Jan. 17, 2019. U.S. patentapplication Ser. No. 16/250,204 is a continuation application of PCTPatent Application No. PCT/CN2017/108468, filed on Oct. 31, 2017, whichclaims priority to Chinese Patent Application No. 201610933471.7,entitled “DATA PROCESSING METHOD AND APPARATUS FOR CONVOLUTIONAL NEURALNETWORK” filed with the Patent Office of China on Oct. 31, 2016, whichis incorporated by reference in its entirety.

FIELD OF TECHNOLOGY

The present disclosure relates to the field of neural networktechnologies and, specifically, to a data processing method andapparatus for a convolutional neural network (CNN).

BACKGROUND

Neural networks and deep learning algorithms have been successfullyapplied and are rapidly developing. It is commonly expected in theindustry that such novel calculation approaches can help implement morecommon and more complex intelligent applications. A convolutional neuralnetwork (CNN) plays an important role in deep learning because of itseffect in the field of images, and is one of the most widely appliedneural networks.

A convolution operation of the CNN is mainly focused on theconvolutional layer, and the convolution operation of the CNN may bedivided into two processes, namely, data expansion and matrixmultiplication. However, during a data expansion process of the CNN,some pieces of data are repeatedly read many times, easily resulting inincrement of data bandwidth or enlargement of storage space required forthe convolution operation, and degradation of the data processingcapability of the processing system for the CNN.

The disclosed methods and systems are directed to solve one or moreproblems set forth above and other problems.

SUMMARY

Embodiments of the present disclosure provide a data processing methodand apparatus for a CNN and a non-volatile computer-readable storagemedium, so as to improve a data processing capability of the processingsystem for the CNN.

One aspect of the present disclosure includes a data processing methodfor a convolutional neural network (CNN). The method includes: (a).obtaining, by a computing device, a matrix parameter of an eigenmatrix;(b). reading, by the computing device, corresponding data in an imagedata matrix from a first buffer space based on the matrix parameterthrough a first bus, to obtain a next to-be-expanded data matrix, andsending and storing the to-be-expanded data matrix to a second presetbuffer space through a second bus; (c). reading, by the computingdevice, the to-be-expanded data matrix from the second preset bufferspace through the second bus, and performing data expansion on theto-be-expanded data matrix based on the matrix parameter, to obtainexpanded data; (d). reading, by the computing device, a preset number ofpieces of unexpanded data in the image data matrix from the first bufferspace through the first bus, sending and storing the unexpanded data tothe second preset buffer space through the second bus, and updating,based on the unexpanded data, the to-be-expanded data matrix stored inthe second preset buffer space; and (e). repeating, by the computingdevice, (c) and (d) until all data in the image data matrix iscompletely read out on the to-be-expanded data matrix based on thematrix parameter.

Another aspect of the present disclosure includes a data processingsystem. The data processing system includes a central processing unit(CPU) configured to process data associated with a convolutional neuralnetwork (CNN); and a co-processor configured to performing operations ofthe CNN, including: (a). obtaining a matrix parameter of an eigenmatrix;(b). reading corresponding data in an image data matrix from a firstbuffer space based on the matrix parameter through a first bus, toobtain a next to-be-expanded data matrix, and sending and storing theto-be-expanded data matrix to a second preset buffer space through asecond bus; (c). reading the to-be-expanded data matrix from the secondpreset buffer space through the second bus, and performing dataexpansion on the to-be-expanded data matrix based on the matrixparameter, to obtain expanded data; (d). reading a preset number ofpieces of unexpanded data in the image data matrix from the first bufferspace through the first bus, sending and storing the unexpanded data tothe second preset buffer space through the second bus, and updating,based on the unexpanded data, the to-be-expanded data matrix stored inthe second preset buffer space; and (e). repeating (c) and (d) until alldata in the image data matrix is completely read out on theto-be-expanded data matrix based on the matrix parameter.

Another aspect of the present disclosure includes a non-transitorycomputer-readable storage medium. The non-transitory computer-readablestorage medium stores computer program instructions executable by atleast one processor to perform: (a). obtaining a matrix parameter of aneigenmatrix; (b). reading corresponding data in an image data matrixfrom a first buffer space based on the matrix parameter through a firstbus, to obtain a next to-be-expanded data matrix, and sending andstoring the to-be-expanded data matrix to a second preset buffer spacethrough a second bus; (c). reading the to-be-expanded data matrix fromthe second preset buffer space through the second bus, and performingdata expansion on the to-be-expanded data matrix based on the matrixparameter, to obtain expanded data; (d). reading a preset number ofpieces of unexpanded data in the image data matrix from the first bufferspace through the first bus, sending and storing the unexpanded data tothe second preset buffer space through the second bus, and updating,based on the unexpanded data, the to-be-expanded data matrix stored inthe second preset buffer space; and (e). repeating (c) and (d) until alldata in the image data matrix is completely read out on theto-be-expanded data matrix based on the matrix parameter.

Other aspects of the present disclosure can be understood by thoseskilled in the art in light of the description, the claims, and thedrawings of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

To more explicitly explain technical solutions in embodiments of thepresent disclosure, accompanying drawings describing the embodiments arebriefly introduced in the following. Apparently, the followingaccompanying drawings are only some embodiments of the presentdisclosure, and a person skilled in the art can derive other drawingsfrom the accompanying drawings without creative efforts.

FIG. 1 is a flowchart of a data processing method for a CNN according toan embodiment of the present disclosure;

FIGS. 2A-2D are schematic diagrams of data sliding and expansionaccording to an embodiment of the present disclosure;

FIGS. 3A-3I are schematic diagrams of data expansion for a CNN;

FIG. 4 is a schematic diagram of matrix multiplication for a CNN;

FIG. 5 is a schematic architectural diagram of a processing system for aCNN according to an embodiment of the present disclosure;

FIG. 6A is a schematic diagram of performing data expansion for a CNN ona CPU;

FIG. 6B is a schematic diagram of performing data expansion for a CNN ona Field Programmable Gate Array (FPGA);

FIGS. 7A-7C are schematic diagrams of data expansion for a CNN accordingto an embodiment of the present disclosure;

FIG. 8A is another flowchart of a data processing method for a CNNaccording to an embodiment of the present disclosure;

FIG. 8B is a schematic diagram of reading and writing a ring bufferaccording to an embodiment of the present disclosure;

FIG. 8C is a schematic architectural diagram of a CNN-based servicescenario according to an embodiment of the present disclosure;

FIG. 9A is a schematic structural diagram of a data processing apparatusfor a CNN according to an embodiment of the present disclosure;

FIG. 9B is another schematic structural diagram of a data processingapparatus for a CNN according to an embodiment of the presentdisclosure; and

FIG. 9C is a schematic structural diagram of a co-processor according toan embodiment of the present disclosure.

DETAILED DESCRIPTION

The following describes the technical solutions in the embodiments ofthe present disclosure with reference to the accompanying drawings.Obviously, the described embodiments are only some embodiments insteadof all embodiments of the present disclosure. Other embodiments obtainedby a person of ordinary skill in the art based on the disclosedembodiments without any creative effort fall within the protection scopeof the present disclosure.

The embodiments of the present disclosure provide a data processingmethod and apparatus for a CNN. Detailed descriptions are separatelyprovided below.

This embodiment is described from the perspective of a data processingapparatus for a CNN. The data processing apparatus may be specificallyintegrated in a processor of a computing device, where the processor maybe a CPU, or integrated in a co-processor such as an FPGA, anapplication-specific integrated circuit (ASIC), a graphics processingunit (GPU).

A data processing method for a CNN includes: obtaining a matrixparameter of an eigenmatrix; reading corresponding data in an image datamatrix based on the matrix parameter, to obtain a to-be-expanded datamatrix; performing data expansion on the to-be-expanded data matrixbased on the matrix parameter, to obtain expanded data; reading a presetnumber of pieces of unexpanded data in the image data matrix, andupdating, based on the unexpanded data, the to-be-expanded data matrix;and returning to performing data expansion on the to-be-expanded datamatrix based on the matrix parameter.

As shown in FIG. 1, a data processing method for a CNN is applied to acomputing device, and a specific process performed by a processor or aco-processor of the computing device may include the followings.

Step 101: Obtaining a matrix parameter of an eigenmatrix.

The eigenmatrix is a convolution kernel of a convolution operation, andis also referred to as a weight matrix. The eigenmatrix may be set basedon actual requirements. The matrix parameter(s) of the eigenmatrix mayinclude numbers of rows and columns of the matrix and may be referred toas the size of the convolution kernel.

Step 102: Reading corresponding data in an image data matrix from afirst buffer space based on the matrix parameter through a first bus, toobtain a to-be-expanded data matrix, and sending and storing theto-be-expanded data matrix to a second preset buffer space through asecond bus.

The elements in the image data matrix are pixel data, such as aprocessed pixel values, corresponding to image pixels. Numbers of rowsand columns of the image data matrix represent the size of an image.

The image data matrix may be stored in an accelerator card of aprocessing system for a CNN. For example, the image data matrix isstored in a DDR memory (double data rate synchronous dynamicrandom-access memory) of an accelerator card, which is hardwareprocessing circuit board for providing additional processing power foraccelerating the data processing. If the image data matrix is stored ina DDR memory, the foregoing first bus is a bus connected between theprocessor or co-processor and the DDR memory. That is, “readcorresponding data in an image data matrix from a first buffer spacebased on the matrix parameter through a first bus, to obtain ato-be-expanded data matrix” in Step 102 may include: read correspondingdata in an image data matrix from a DDR memory based on the matrixparameter through a bus connected between the processor or co-processorand the DDR memory.

In one embodiment, a matrix with a corresponding number of rows or acorresponding number of columns in the image data matrix may be read outbased on the matrix parameter(s).

When the matrix parameter includes numbers of rows and columns of theeigenmatrix, a read-out number of rows may correspond to the number ofrows of the eigenmatrix, or a read-out number of columns may correspondto the number of columns of the eigenmatrix.

For example, when the image data matrix is an N*N matrix, and theeigenmatrix is a K*K matrix, K rows of data in the N*N image data matrixmay be read out, to obtain a K*N to-be-expanded data matrix. K and N arepositive integers, and K≤N.

A starting location of the read-out data may be set based on actualrequirements. For example, the K rows of data may be read out startingfrom a first row of the image data matrix, or the K rows of data may beread out starting from a second row.

For another example, when the image data matrix is an N*N matrix, andthe eigenmatrix is a K*M matrix, M columns of data in the N*N image datamatrix may be read out, to obtain an N*M to-be-expanded data matrix. Mis a positive integer, and M≤N.

After the to-be-expanded data matrix is obtained, the to-be-expandeddata matrix may be sent and stored to a second preset buffer spacethrough a second bus. The second preset buffer space may be a presetbuffer. For example, the preset buffer may be a buffer or a DDR memoryin the co-processor, and the second bus is a bus connected between theprocessor or the co-processor and the preset buffer.

Step 103: Reading the to-be-expanded data matrix from the second presetbuffer space through the second bus, and performing data expansion onthe to-be-expanded data matrix based on the matrix parameter, to obtainexpanded data;

Specifically, data expansion may be performed on the to-be-expanded datamatrix based on the numbers of rows and columns of the eigenmatrix.After the expansion, several data sets can be obtained. After the dataexpansion on the image data matrix is completed, a data matrix, that is,an expanded data matrix, may be formed based on the data sets.Subsequently, matrix multiplication may be performed based on theexpanded data matrix and the eigenmatrix, to obtain corresponding dataand complete a convolution operation of data.

For example, after the K*N to-be-expanded data matrix is obtained, theK*N to-be-expanded data matrix may be expanded based on the numbers ofrows and columns of the eigenmatrix.

In this case, the process of performing data expansion on theto-be-expanded data matrix based on the matrix parameter may include:performing data expansion on the to-be-expanded data matrix based on thematrix parameter and a storage address of data of the to-be-expandeddata matrix in the second preset buffer space.

For example, the K*N to-be-expanded data matrix is written to the secondpreset buffer space, and then data expansion is performed on the K*Nto-be-expanded data matrix based on numbers of rows and columns of theK*K eigenmatrix and a storage address of data of the K*N to-be-expandeddata matrix in the second preset buffer space.

In one embodiment, sliding and data expansion may be performed on theto-be-expanded data matrix. Specifically, a window is slid on theto-be-expanded data matrix, data expansion is performed on data in thewindow after each time of sliding, and several data sets may be obtainedby the expansion. That is, the process of performing data expansion onthe to-be-expanded data matrix based on the matrix parameter and astorage address of data of the to-be-expanded data matrix in the secondpreset buffer space may include: determining a sliding window based onthe matrix parameter; moving the sliding window on the to-be-expandeddata matrix based on a preset sliding direction and a preset slidingstep length; obtaining a storage address of data in the sliding windowin the second preset buffer space after each time of sliding; andreading the corresponding data from the second preset buffer space basedon the storage address, to complete the data expansion.

Specifically, a sliding window of a corresponding size can be determinedbased on row and column data of the eigenmatrix. For example, when theeigenmatrix is a K*K matrix, a K*K sliding window can be determined. Thesliding window may be used for selecting corresponding data from theto-be-expanded data matrix for expansion.

The preset sliding direction may include: a row direction, a columndirection, and the like, of the image data matrix. In actualapplications, the preset sliding direction may correspond to a datareading manner in Step 102. For example, when several rows of data inthe image data matrix are read out, the preset sliding direction may bethe row direction of the image data matrix. For another example, whenseveral columns of data in the image data matrix are read out, thepreset sliding direction may be the column direction of the image datamatrix.

The preset sliding step length is the required sliding distance, can beset based on actual requirements for data expansion, and can berepresent by the number of pieces of data need to be slid in the datamatrix. For example, the preset sliding step length is one piece, twopieces, or three pieces of data, or the like.

After the preset sliding step length, the preset sliding direction, andthe sliding window are obtained, the sliding window may be slid or movedon the to-be-expanded data matrix along the preset sliding direction atthe preset sliding step length. After each time the window is slid, anaddress of data in the window in the second preset buffer space can beobtained and, then, corresponding data is read from a preset bufferbased on the address and a preset reading sequence, to complete dataexpansion. That is, data expansion is implemented by reading data in anaddress hopping manner.

In a schematic diagram of data sliding and expansion as shown in FIG. 2Ato FIG. 2D, reading corresponding rows of data from the image datamatrix as is used an example, and referring to FIG. 2A, it is assumedthat the image data matrix is a 5*5 matrix, and the eigenmatrix is a 3*3matrix. First, three rows of data are read from the 5*5 image datamatrix to obtain a 3*5 to-be-expanded data matrix, that is, the matrixin FIG. 2B to FIG. 2D, and are written into the second preset bufferspace, and then, a sliding window, that is, a dashed-line box, isdetermined based on numbers of rows and columns of the 3*3 eigenmatrix.Referring to FIG. 2B to FIG. 2D, the sliding window may be slid on the3*5 to-be-expanded data matrix along the row direction at the slidingstep length of one piece of data, that is, the sliding window is slidfrom left to right.

Referring to FIG. 2B, at an initial sliding location, that is, after the0^(th) sliding, a storage address of data in the sliding window in thesecond preset buffer space can be obtained, and then corresponding datais read from the second preset buffer space in an address hopping mannerbased on the storage address, to obtain a data set (11, 12, 13, 21, 22,23, 31, 32, 33), which is referred to as a first data set. Referring toFIG. 2C, after the first data set is obtained, the window is slid in therow direction at the sliding step length of one piece of data. Then, astorage of data in the sliding window in the second preset buffer spaceis obtained, and corresponding data is read from the second presetbuffer space in an address hopping manner based on the storage address,to obtain a data set (12, 13, 14, 22, 23, 24, 32, 33, 34), which isreferred to as a second data set.

Referring to FIG. 2D, after the second data set is obtained, the windowis continuously slid in the row direction at the sliding step length ofone piece of data and, then, a storage of data in the sliding window inthe second preset buffer space is obtained. Further, corresponding datais read from the second preset buffer space in an address hopping mannerbased on the storage address, to obtain a data set (13, 14, 15, 23, 24,25, 33, 34, 35), which is referred to as a third data set. By then, dataexpansion on the 3*5 to-be-expanded data matrix is completed.

In one embodiment, the initial location of the sliding window on theto-be-expanded data matrix can be set based on actual requirements, Forexample, referring to FIG. 2B, sliding may be started from a firstcolumn of the to-be-expanded data matrix. In another embodiment, slidingmay alternatively be started from a second or third column of theto-be-expanded data matrix.

Similarly, when a corresponding column of data is read from the imagedata matrix to form a to-be-expanded data matrix, the sliding window mayalternatively be determined based on numbers of rows and columns of theeigenmatrix. Then, the window is slid along a column direction of theto-be-expanded data matrix at a preset sliding step length, and aftereach time the window is slid, a storage address of data in the window inthe second preset buffer space is obtained, and corresponding data isread from the second preset buffer space based on the storage address.The data sliding and expansion process thereof is similar to the datasliding and expansion introduced in the foregoing embodiment. Refer toFIG. 2A to 2D. Details are not described herein again.

Step 104: Reading out a preset number of pieces of unexpanded data inthe image data matrix from the first buffer space through the first bus,sending and storing the unexpanded data to the second preset bufferspace through the second bus, and updating, based on the unexpandeddata, the to-be-expanded data matrix stored in the second preset bufferspace; and returning to Step 103.

Specifically, a preset number of pieces of unexpanded data in the imagedata matrix are read out from the first buffer space through the firstbus, the read-out unexpanded data is sent and stored to the secondpreset buffer space through the second bus, and the to-be-expanded datamatrix stored in the second preset buffer space is updated based on theunexpanded data.

The number of pieces of the unexpanded data may be set, based on actualrequirements, to be, for example, one piece or five pieces, one row ortwo rows, or one column or two columns.

Specifically, the preset number of pieces of unexpanded data in theimage data matrix may be read out from the first buffer space based on aconvolution step length through the first bus. The convolution steplength represents a number of rows or a number of columns of theunexpanded data that needs to be read from the image data matrix afterthe to-be-expanded data matrix is expanded.

In an example in which the image data matrix is an N*N matrix, and theeigenmatrix is a K*K matrix, after data expansion is performed on theK*N to-be-expanded data matrix, a specific number of pieces of data,such as a corresponding number of rows or columns of unexpanded data,can be read out from the N*N image data matrix stored in the firstbuffer space based on the convolution step length. For example, when theconvolution step length S=1, a row or a column of unexpanded data can beread from the N*N image data matrix based on the convolution step lengthand, then, the to-be-expanded data matrix stored in the second presetbuffer space is updated based on the read-out unexpanded data.

Specifically, reading row data from the image data matrix to form ato-be-expanded data matrix is used as an example. When the convolutionstep length S=1, after data expansion is performed on the to-be-expandeddata matrix, the (K+1)^(th) row of data can be read from the image datamatrix stored in the first buffer space, and the to-be-expanded datamatrix stored in the second preset buffer space is updated based on the(K+1)^(th) row of data. After the to-be-expanded data matrix is updated,Step 102 is performed again to read the updated to-be-expanded datamatrix from the second preset buffer space through the second bus, anddata expansion is performed on the updated to-be-expanded data matrix.

After the expansion is completed, the (K+2)^(th) row of data is furtherread from the image data matrix stored in the first buffer space, andthe current to-be-expanded data matrix is updated based on the(K+2)^(th) row of data. Step 102 is performed again to read the updatedto-be-expanded data matrix from the second preset buffer space throughthe second bus and perform data expansion on the updated to-be-expandeddata matrix. After the expansion is completed, the (K+3)^(th) row ofdata is further read from the image data matrix stored in the firstbuffer space, and so on until all pieces of data in the image datamatrix are completely read out.

The process of updating, based on the unexpanded data, theto-be-expanded data matrix stored in the second preset buffer space mayinclude: reading the unexpanded data from the second preset buffer spacethrough the second bus, and selecting a preset number of pieces oftarget data from the unexpanded data; and updating, based on the targetdata, the to-be-expanded data matrix stored in the second preset bufferspace. For example, when at least two rows or two columns of unexpandeddata in the image data matrix are read, the to-be-expanded data matrixmay be updated by selecting one row or column of data from the two rowsor two columns of data.

In another embodiment, if the number of pieces of the read-outunexpanded data is equal to the preset number corresponding to thetarget data, the to-be-expanded data matrix can be updated directlybased on the unexpanded data. For example, when the preset numbercorresponding to the target data is a number of pieces of a row of data,after a row of data in the image data matrix is read, the to-be-expandeddata matrix is updated directly based on the row of data. For example,after the (K+1)^(th) row of data is read, the to-be-expanded data matrixis updated directly based on the (K+1)^(th) row of data.

In one embodiment, a manner of updating the to-be-expanded data matrixmay include a data-overwriting manner. That is, corresponding data inthe to-be-expanded data matrix is overwritten based on the selectedtarget data, to complete the update.

In one embodiment, because some pieces of data in the to-be-expandeddata matrix that have been stored into the second preset buffer spacecan be multiplexed, it is only needed to read a preset number ofunexpanded data, that is, data that is not stored to the second presetbuffer space, in the image data matrix from the first buffer space, soas to avoid repeatedly reading some pieces of data during data expansionand reduce storage space in a processing system for a CNN. Moreover,because it is only needed to read a preset number of pieces ofunexpanded data in the image data matrix through the first bus, and sendthe unexpanded data to the second preset buffer space through the secondbus, the volume of transmitted data is reduced, and transmissionbandwidth of the first bus and the second bus is saved, thereby furtherimproving a data processing capability of the system. Specificimplementation processes of the data processing method in several typesof convolution operations in the embodiments of the present disclosureare introduced below.

Using the image data matrix and eigenmatrix shown in FIG. 2A as anexample, FIG. 3A to FIG. 3I show a process of performing data expansionby using a data processing method.

As shown in FIG. 3A, the first group of data is expanded; in FIG. 3B,the second group of data is expanded; in FIG. 3C, the third group ofdata is expanded; in FIG. 3D, the fourth group of data is expanded; inFIG. 3E, the fifth group of data is expanded; in FIG. 3F, the sixthgroup of data is expanded; in FIG. 3G, the seventh group of data isexpanded; in FIG. 3H, the eighth group of data is expanded; and in FIG.3I, the ninth group of data is expanded.

After the data expansion, multiplication may be performed on theexpanded data matrix and a convolution kernel by referring to FIG. 4, tocomplex a convolution operation.

It could be learned from FIG. 3A to FIG. 3I that in the data processingmethod, during data expansion, some pieces of data are repeatedly readmany times, easily resulting in increment of data bandwidth orenlargement of storage space, and degradation of a data processingcapability of a processing system.

In an actual situation, if the processing system for a CNN uses the dataexpansion manner shown in FIG. 3A to FIG. 3I, data transmissionbandwidth required for data expansion is increased, storage space isenlarged, and a data processing capability of the processing system isdegraded.

The processing system for a CNN shown in FIG. 5 is used as an example todescribe a specific implementation process by using a data expansionmanner shown in FIG. 3A to FIG. 3I. The processing system includes aco-processor, a CPU of a server and a memory unit (memory), and a DDRmemory on an accelerator card. The CPU of the server is connected to theco-processor usually through a peripheral component interconnect expressPCI-e (a bus and interface standard) bus, to perform data exchange andcommand exchange, for example, to perform command exchange by using acommand path (CMD path) and to perform data exchange by using a datapath.

The co-processor may be an FPGA or another auxiliary processor, and theco-processor may include: a DDR memory controller, an input buffer(InputBuf) (an input data buffer unit), an output buffer (OutputBuf) (anoutput data buffer unit), and a processing element (PE). The PE is anelement configured to complete data convolution in the co-processor.

Data expansion in a current convolution operation may be completed in aCPU of the processing system, or the co-processor as follows.

In an embodiment of the present disclosure, data expansion is completedby using the CPU of the system.

Referring to FIG. 6A, a solution of data expansion by using a CPUincludes: expanding, by a processing system CPU, data in the manner inFIG. 3A to FIG. 3I; after completing expansion, storing the expandeddata to a CPU memory, and transmitting the expanded data to a DDR memoryon an accelerator card through a PCI-e DMA; and further loading, by theco-processor, the data from the DDR memory on the accelerator card tothe PE by using loading logic. It could be learned from FIG. 6A that, ifthe data processing method shown in FIG. 3A to FIG. 3I is used in theCPU of the system, consequently, data expansion efficiency is relativelylow and a volume of transmitted data is increased. Consequently, neededreading bandwidth of the PCI-e and the DDR memory is increased, and aprocessing capability of the system is degraded.

However, if the data processing method shown in FIG. 2B to FIG. 2D ofone embodiment is applied to the CPU of the system, because it is notneeded to read-out data repeatedly, data expansion efficiency can beimproved. If the data processing method shown in FIG. 2B to FIG. 2D ofone embodiment is applied to the co-processor of the system, neededtransmission bandwidth of the PCI-e and the DDR memory is reduced.

Thus, in another embodiment of the present disclosure, data expansion iscompleted by using the co-processor of the system.

Referring to FIG. 6B, a solution of performing data expansion by usingthe co-processor includes: storing the unexpanded data to a servermemory, an accelerator card DDR memory, and an FPGA, and expanding, bythe FPGA, data in the manner of FIG. 3A to FIG. 3I. As can be learnedfrom FIG. 6B, because in this solution, data expansion is performed byusing the manner shown in FIG. 3A to FIG. 3I, some pieces of data arerepeatedly read, resulting in relatively low efficiency of dataexpansion, a large volume of transmitted DDR memory data, and a lot ofstorage units on the FPGA chip that need to be consumed.

The system shown in FIG. 5 and the image data matrix and the eigenmatrixshown in FIG. 5 are used as an example to introduce the data processingmethod provided by one embodiment of the present disclosure. It isassumed that the convolution step length S=1, and the data processingapparatus of one embodiment is integrated in the co-processor.

Referring to FIG. 7A to FIG. 7C, the image data matrix is stored in theDDR memory on the accelerator card. Specifically, the data expansionprocess is as follows:

(1) K=3 rows of data [11, 12, 13, 14, 15], [21, 22, 23, 24, 25], and[31, 32, 33, 34, 35] in the image data matrix are read, to obtain ato-be-expanded data matrix, and the to-be-expanded data matrix is loadedin to the memory of the co-processor, as shown in FIG. 7A.

(2) Data sliding and expansion is performed on the to-be-expanded datamatrix in the memory, to obtain expanded data [11, 12, 13, 21, 22, 23,31, 32, 33], [12, 13, 14, 22, 23, 24, 32, 33, 34], and [13, 14, 15, 23,24, 25, 33, 34, 35], as shown in FIG. 7A.

Specifically, a sliding window can be determined based on row and columndata of the eigenmatrix, then, the window is slid along a row directionof the to-be-expanded data matrix at a sliding step length of one pieceof data, and after each time the window is slid, corresponding data isread from the memory in an address hopping manner based on a storageaddress of data in the window in the memory, to implement dataexpansion.

(3) The fourth row of data [41, 42, 43, 44, 45] in the image data matrixis loaded to the memory, and the first row of data [11, 12, 13, 14, 15](that is, a row of data currently having an earliest storage time) inthe current to-be-expanded data matrix is overwritten, to update theto-be-expanded data matrix, as shown in FIG. 7B.

(4) Data sliding and expansion is performed on the updatedto-be-expanded data matrix, to obtain expanded data [21, 22, 23, 31, 32,33, 41, 42, 43], [22, 23, 24, 32, 33, 34, 42, 43, 44], and [23, 24, 25,33, 34, 35, 43, 44, 45], as shown in FIG. 7B.

(5) The fifth row of data [51, 52, 53, 54, 55] in the image data matrixis loaded to the memory, and the first row of data [21, 22, 23, 24, 25]in the current to-be-expanded data matrix is overwritten, to update theto-be-expanded data matrix, as shown in FIG. 7C.

(6) Data sliding and expansion is performed on the updatedto-be-expanded data matrix, to obtain expanded data [31, 32, 33, 41, 42,43, 51, 52, 53], [32, 33, 34, 42, 43, 44, 52, 53, 54], [33, 34, 35, 43,44, 45, 53, 54, 55], as shown in FIG. 7C.

Based on the foregoing description of the data expansion solution, thedata expansion solution shown in FIG. 7A to FIG. 7C can improve dataexpansion efficiency. In addition, in one embodiment of the presentdisclosure, data expansion is performed in the co-processor, and dataexpansion is performed in a multiplexed reading manner, the volume ofread-out data can be reduced, thereby reducing requirements for readingbandwidth of the PCI-e and DDR memory and improving the processingcapability of the system.

Accordingly, the data expansion solution shown in FIG. 7A to FIG. 7C canimprove data expansion efficiency and can also save storage space of theco-processor, thereby reducing requirements of data expansion oncorresponding storage space and improving the processing capability ofthe system.

To improve efficiency of reading data from the DDR memory and dataexpansion efficiency, in one embodiment, a preset number of pieces ofdata in the image data matrix can be read based on a fixed data size.That is, the process of reading a preset number of pieces of unexpandeddata in the image data matrix may include: reading a preset number ofpieces of unexpanded data in the image data matrix based on a firstpredetermined data volume; and storing the unexpanded data to the presetbuffer space.

The first predetermined data volume may be set based on actualrequirements, for example, 8 Kbyte, 16 Kbyte, or the like. Theunexpanded data of the first predetermined data volume may be referredto as a data packet (packet). The first predetermined data volume may beset based on row data or column data of the image data matrix, forexample, may be an integer multiple of the data volume of the row dataor column data.

To improve data expansion efficiency and a utilization ratio of a bufferspace, in one embodiment of the present disclosure, when remaining spaceof the preset buffer space is sufficient for loading a new packet, datais read out and loaded. That is, the process of reading a preset numberof pieces of unexpanded data in the image data matrix based on a firstpredetermined data volume may include: obtaining a remaining availablecapacity of the preset buffer space; and when the remaining availablecapacity of the preset buffer space is greater than or equal to thefirst predetermined data volume, reading the preset number of pieces ofunexpanded data in the image data matrix based on the firstpredetermined data volume.

Because the first predetermined data volume is set based on actualrequirements, and a packet loaded each time is greater than a datavolume of a row or a column of data in the image data matrix, after anew packet is loaded, a specific number of pieces of target data can beselected from the packet to update the to-be-expanded data matrix. Thatis, the process of updating the to-be-expanded data matrix based on theunexpanded data may include: selecting a preset number of pieces oftarget data from the unexpanded data; and updating the to-be-expandeddata matrix based on the target data.

Specifically, data belonging to the same row or same column in the imagedata matrix may be selected. For example, the first predetermined datavolume is a data volume of 8 pieces of data. That is, a packet includes8 pieces of data in the image data matrix. After expansion on theto-be-expanded data matrix is completed, 8 pieces of unexpanded data maybe read from the image data matrix and are assumed to be [41, 42, 43,44, 45, 51, 52, 53] in FIG. 7A. Subsequently, target data located on asame row, that is, [41, 42, 43, 44, 45], is selected from the 8 piecesof unexpanded data, and the to-be-expanded data matrix is updated basedon the target data. In addition, when the target data is selected, arelationship with the last row or last column of data of theto-be-expanded data matrix also needs to be considered.

To improve a data expansion speed, in the method of one embodiment, dataexpansion can be performed when data currently buffered by the presetbuffer space is sufficient for data expansion. That is, the process ofperforming data expansion on the to-be-expanded data matrix based on thematrix parameter and a storage address of data of the to-be-expandeddata matrix in the preset buffer space may include: obtaining a currentbuffer data volume of the preset buffer space; and when the buffer datavolume is greater than or equal to the second predetermined data volume,performing data expansion on the to-be-expanded data matrix based on thematrix parameter and the storage address of the data of theto-be-expanded data matrix in the preset buffer space.

The second predetermined data volume may be determined based on numbersof rows and columns of the eigenmatrix and numbers of rows and columnsof the image data matrix. In an example where the image data matrix isan N*N matrix, and the eigenmatrix is a K*K matrix, the second presetdata volume may be a data volume of K*N pieces of data.

Accordingly, one embodiment of the present disclosure uses the followingsteps: obtaining a matrix parameter of an eigenmatrix; then readingcorresponding data in an image data matrix from based on the matrixparameter, to obtain a to-be-expanded data matrix; performing dataexpansion on the to-be-expanded data matrix based on the matrixparameter, to obtain expanded data; reading a preset number of pieces ofunexpanded data in the image data matrix, and updating, based on theunexpanded data, the to-be-expanded data matrix; and returning to thestep of performing expansion on the to-be-expanded data matrix based onthe matrix parameter. In this solution, during a convolution process,the read-out image data can be multiplexed to implement data expansion,so as to avoid repeatedly reading of some pieces of data, and reducerequirements of CNN data expansion for data bandwidth or storage space.Therefore, a data processing capability and data expansion efficiency ofa processing system for a CNN can be improved.

According to the method described in the embodiments shown in FIG. 1, adata processing method for a CNN is further described in detail below.

In one embodiment, descriptions are provided by using the dataprocessing apparatus for a CNN integrated in a co-processor of acomputing device and a system architecture shown in FIG. 5. Theco-processor may be an FPGA, an ASIC, or a co-processor of another type.In one embodiment, an image data matrix is stored in a DDR memory of aprocessing system.

As shown in FIG. 8A, a data processing method for CNN may include thefollowing specific procedure.

Step 201: A co-processor obtains a system parameter, where the systemparameter includes a matrix parameter of an eigenmatrix.

The matrix parameter may include numbers of rows and columns of theeigenmatrix. In one embodiment, the system parameter may further includenumbers of rows and columns of an image data matrix, a predetermineddata volume B, a predetermined data volume A, a sliding direction, asliding step length, and the like.

Step 202: The co-processor reads a corresponding number of rows of datafrom a DDR memory based on the matrix parameter of the eigenmatrix, toobtain a to-be-expanded data matrix Q.

For example, K rows of data of an N*N image data matrix are read fromthe DDR memory, to obtain a K*N to-be-expanded data matrix Q.Specifically, the first to K^(th) rows of data of the N*N image datamatrix are read.

Using the 5*5 image data matrix and the 3*3 eigenmatrix shown in FIG. 2Aas an example, an FPGA can read first to third rows of data from the 5*5image data matrix, to form a 3*5 to-be-expanded data matrix Q.

Step 203: The co-processor writes the to-be-expanded data matrix Q intoa buffer of the co-processor. For example, the FPGA writes the 3*5to-be-expanded data matrix Q into a buffer in the FPGA.

Step 204: When the volume of data currently buffered in the buffer isgreater than a predetermined data volume A, the co-processor performsdata sliding and expansion on the data matrix Q based on the matrixparameter of the eigenmatrix, to obtain expanded data.

The predetermined data volume A may be a data volume of 3*5 pieces ofdata, and can be specifically set based on actual requirements.

In one embodiment, the buffer may be a ring buffer. Referring to FIG.8B, the ring buffer has two indicators. One is LenBufSpaceReady, usedfor representing a remaining space or a remaining available capacity ofthe ring buffer. The other is LenBufDataValid, used for presenting abuffer data volume currently buffered by the ring buffer.

After data is written, LenBufSpaceReady is reduced by 1, andLenBufDataValid is increased by 1. When expansion is performed and datais read, LenBufSpaceReady is increased by 1, and LenBufDataValid isreduced by 1. In one embodiment, data can be concurrently loaded andwritten, and expanded and read, to improve data expansion efficiency.

When determining that LenBufDataValid is greater than the predetermineddata volume A, the co-processor performs data sliding and expansion onthe data matrix Q based on the matrix parameter of the eigenmatrix.Otherwise, data sliding and expansion is not performed.

Using the 5*5 image data matrix and the 3*3 eigenmatrix shown in FIG. 2Aas an example, the FPGA may perform data sliding and expansion on a 3*5to-be-expanded data matrix Q in a manner shown in FIG. 2B to FIG. 2D, toobtain expanded data (11, 12, 13, 21, 22, 23, 31, 32, 33), (12, 13, 14,22, 23, 24, 32, 33, 34), and (13, 14, 15, 23, 24, 25, 33, 34, 35).

Step 205: When the remaining available capacity of the buffer is greaterthan a predetermined data volume B, the co-processor reads acorresponding number of pieces of unexpanded data from the DDR memorybased on the predetermined data volume B, and writes the unexpanded datato the buffer.

For example, when determining that LenBufSpaceReady is greater than thepredetermined data volume B, the co-processor reads the predetermineddata volume B of unexpanded data from the DDR memory, and writes theunexpanded data to the buffer.

The predetermined data volume B is a fixed data volume, that is, a fixeddata size, and can be set based on actual requirements. For example, thepredetermined data volume B may be 8 Kbyte or the like, and thepredetermined data volume B may be set based on a data volume of a rowor a column of data in the image data matrix.

For example, referring to FIG. 7A and FIG. 7B, after data sliding andexpansion is performed on a 3*3 to-be-expanded data matrix, the fourthrow of unexpanded data [41, 42, 43, 44, 45] can be read and written tothe buffer.

In one embodiment, when the predetermined data volume B is a data volumeof a row or a column of data in the image data matrix, that is, when anumber of pieces of image data corresponding to the predetermined datavolume B is equal to a number of columns and a number of rows of thematrix, the co-processor can read a row or a column of unexpanded datain the image data matrix from the DDR memory. For example, the(K+1)^(th) row of unexpanded data, that is, N pieces of unexpanded data,can be read.

In another embodiment, a number of pieces of image data corresponding tothe predetermined data volume B is greater than N, but is not an integermultiple of N, for example, may be N+1 or the like. For example, afterdata sliding and expansion is performed on a 3*3 to-be-expanded datamatrix, seven pieces of unexpanded data [41, 42, 43, 44, 45, 51, 52] canbe read and written to the buffer based on the predetermined data volumeB.

Step 206: The co-processor updates the to-be-expanded data matrix Qbased on the written unexpanded data; and the method returns to Step204.

For example, after the (K+1)^(th) row of unexpanded data is written, theto-be-expanded data matrix Q can be updated based on the (K+1)^(th) rowof unexpanded data. For example, the first row of data of the matrix Qis overwritten based on the (K+1)^(th) row of data.

For example, after the (N+1)^(th) row of unexpanded data is written,corresponding N pieces of unexpanded data can be selected and, then, thefirst row of data of the matrix Q is overwritten based on the selected Npieces of unexpanded data.

The data processing method of one embodiment of the present disclosureis applied to all services that can be implemented by a heterogeneousprocessing system using an FPGA as a co-processor or a pure CPUprocessing system. For example, the method may be applied to a servicescenario whose objective is detecting and screening erotic pictures.Referring to FIG. 8C, an open source deep learning platform, such asCaffe or TensorFlow, is used for implementation. When a CNN model (suchas AlexNet, GoogleNet, or VGG) is implemented, the learning platforminvokes a basic linear algebra subprograms (BLAS) library to performmatrix operations. In a pure-CPU processing system the matrix operationsare calculated by a CPU. In a heterogeneous processing system, thematrix operations may be offloaded to the FPGA for calculation (performexchange generally through the PCI-e). During a calculation process, theCPU and FPGA perform data exchange in a DDR memory sharing manner.

Accordingly, one embodiment of the present disclosure uses aco-processor to obtain a matrix parameter of an eigenmatrix; then readcorresponding data in an image data matrix from based on the matrixparameter, to obtain a to-be-expanded data matrix; perform dataexpansion on the to-be-expanded data matrix based on the matrixparameter, to obtain expanded data; read a preset number of pieces ofunexpanded data in the image data matrix, and update, based on theunexpanded data, the to-be-expanded data matrix; and return to the stepof performing expansion on the to-be-expanded data matrix based on thematrix parameter. In this solution, during a convolution process, theread-out image data can be multiplexed to implement data expansion, soas to avoid repeatedly reading of some pieces of data, and reducerequirements of CNN data expansion for data bandwidth or storage space.Therefore, a data processing capability and data expansion efficiency ofa processing system for a CNN can be improved.

To implement the foregoing method better, the embodiments of the presentdisclosure further provide a data processing apparatus for a CNN. Thedata processing apparatus may be specifically integrated in a processorof a computing device. The processor may be an FPGA, an ASIC, a GPU, ora co-processor of another type. As shown in FIG. 9A, the data processingapparatus for a CNN may include an obtaining unit 301, a reading unit302, a data expansion unit 303, and an updating unit 304 as follows:

The obtaining unit 301 is configured to obtain a matrix parameter of aneigenmatrix.

The eigenmatrix is a convolution kernel of a convolution operation, andis also referred to as a weight matrix. The eigenmatrix may be set basedon actual requirements. The matrix parameter(s) of the eigenmatrix mayinclude numbers of rows and columns of a matrix and may be referred toas a size of the convolution kernel.

The reading unit 302 is configured to read corresponding data in animage data matrix from a first buffer space based on the matrixparameter through a first bus, to obtain a to-be-expanded data matrix.

An element in the image data matrix is pixel data, such as a processedpixel value, corresponding to an image pixel. Numbers of rows andcolumns of the image data matrix represent the size of an image.

For example, the reading unit 302 is configured to read a matrix with acorresponding number of rows or a corresponding number of columns in theimage data matrix from the first buffer space based on the matrixparameter through the first bus.

When the matrix parameter includes numbers of rows and columns of theeigenmatrix, a read-out number of rows may correspond to the number ofrows of the eigenmatrix, or a read-out number of columns may correspondto the number of columns of the eigenmatrix.

The storage unit 305 is configured to after the reading unit 302 obtainsthe to-be-expanded data matrix, and before the data expansion unit 303performs data expansion, send and store the to-be-expanded data matrixto a second preset buffer space through a second bus.

The data expansion unit 303 is configured to read the to-be-expandeddata matrix from the second preset buffer space through the second bus,and perform data expansion on the to-be-expanded data matrix based onthe matrix parameter, to obtain expanded data.

For example, the data expansion unit 303 is configured to perform datasliding and expansion on the to-be-expanded data matrix based on thematrix parameter.

The data expansion unit is specifically configured to perform dataexpansion on the to-be-expanded data matrix based on the matrixparameter and a storage address of data of the to-be-expanded datamatrix in the second preset buffer space.

Specifically, the data expansion unit 303 may include: a determiningsubunit, a sliding subunit, an address obtaining subunit, and a readingsubunit.

The determining subunit is configured to determine a sliding windowbased on the matrix parameter. The sliding subunit is configured toslide the sliding window on the to-be-expanded data matrix based on apreset sliding direction and a preset sliding step length. The addressobtaining subunit is configured to obtain a storage address of data inthe sliding window in the second preset buffer space after each time ofsliding. The reading subunit is configured to read the correspondingdata from the second preset buffer space based on the storage address,to complete the data expansion.

The determining subunit can be configured to determine a sliding windowof a corresponding size based on row and column data of the eigenmatrix.For example, when the eigenmatrix is a K*K matrix, a K*K sliding windowcan be determined. The sliding window may be used for selectingcorresponding data from the to-be-expanded data matrix for expansion.

The preset sliding direction may include: a row direction, a columndirection, and the like of the image data matrix. The preset slidingstep length is a needed sliding distance, can be set based on actualrequirements for data expansion, and can be represent by a number ofpieces of data need to be slid in the data matrix. For example, thepreset sliding step length is one piece, two pieces, or three pieces ofdata, or the like.

The sliding subunit may be specifically configured to slide or move thesliding window on the to-be-expanded data matrix along a preset slidingdirection at a preset sliding step length. In one embodiment, theinitial location of the sliding window on the to-be-expanded data matrixcan be set based on actual requirements, For example, referring to FIG.2B, sliding may be started from a first column of the to-be-expandeddata matrix. In another embodiment, sliding may alternatively be startedfrom a second or third column of the to-be-expanded data matrix.

The updating unit 304 is configured to read a corresponding number ofpieces of unexpanded data in the image data matrix from the first bufferspace through the first bus, send and store the unexpanded data to thesecond preset buffer space through the second bus, and update, based onthe unexpanded data, the to-be-expanded data matrix stored in the secondpreset buffer space; and trigger the data expansion unit 303 to performthe step of reading the to-be-expanded data matrix from the secondpreset buffer space through the second bus, and performing expansion onthe to-be-expanded data matrix based on the matrix parameter.

For example, the updating unit 304 may include: a reading subunit, anupdating subunit, and a triggering subunit.

The reading subunit is configured to read the corresponding number ofpieces of unexpanded data in the image data matrix from the first bufferspace based on a first predetermined data volume through the first bus,and send and store the unexpanded data to the second preset buffer spacethrough the second bus. The updating subunit is configured to update,based on the unexpanded data, the to-be-expanded data matrix stored inthe second preset buffer space. The triggering subunit is configured totrigger, after the updating subunit updates the to-be-expanded datamatrix, the data expansion unit 303 to perform the step of reading theto-be-expanded data matrix from the second preset buffer space throughthe second bus, and performing expansion on the to-be-expanded datamatrix based on the matrix parameter.

The reading subunit is specifically configured to: obtain a remainingavailable capacity of the preset buffer space; and when the remainingavailable capacity of the second preset buffer space is greater than orequal to the first predetermined data volume, read the preset number ofpieces of unexpanded data in the image data matrix from the first bufferspace based on the first predetermined data volume through the firstbus.

The updating subunit is specifically configured to: read the unexpandeddata from the second preset buffer space through the second bus, andselect a preset number of pieces of target data from the unexpandeddata; and update, based on the target data, the to-be-expanded datamatrix stored in the second preset buffer space.

In one embodiment, the data expansion unit 303 may be specificallyconfigured to: obtain a current buffer data volume of the preset bufferspace; and when the buffer data volume is greater than or equal to thesecond predetermined data volume, perform data expansion on theto-be-expanded data matrix based on the matrix parameter and the storageaddress of the data of the to-be-expanded data matrix in the secondpreset buffer space.

During specific implementation, the foregoing units may be implementedas independent entities, or may be combined arbitrarily, or may beimplemented as a same entity or several entities. For specificimplementation of the foregoing units, refer to the foregoing methodembodiments. Details are not described herein again.

For example, in actual applications, functions of the obtaining unit 301can be implemented by a data expansion controller, functions of thereading unit 302 can be implemented by a data expansion controller and aDDR memory data reading controller, functions of the data expansion unit303 may be implemented by a data expansion controller, a data scanningcontroller, and an address generator, and functions of the updating unit304 may be implemented by a data expansion controller and a DDR memorydata reading controller.

As shown in FIG. 9C, one embodiment further provides a co-processor,including: a data expansion controller 401, a DDR memory data readingcontroller 402, a data buffer unit 403, a data scanning controller 404,an address generator 405, and a PE 406.

The data expansion controller 401 is configured to obtain a matrixparameter of an eigenmatrix, control the DDR memory data readingcontroller 402 to read corresponding data in an image data matrix basedon the matrix parameter, to obtain a to-be-expanded data matrix, andwrite the to-be-expanded data matrix to the data buffer unit 403.

The data expansion controller 401 is further configured to control,based on a system parameter (for example, the matrix parameter of theeigenmatrix), the data scanning controller 404 and the address generator405 to perform data expansion on the to-be-expanded data matrix, toobtain expanded data; control the DDR memory data reading controller 402to read a preset number of pieces of unexpanded data in the image datamatrix, and control the DDR memory data reading controller 402 to updatethe to-be-expanded data matrix based on the unexpanded data; and triggerthe data scanning controller 404 and the address generator 405 toperform expansion on the to-be-expanded data matrix.

For example, the data expansion controller 401 may control, based on thesystem parameter (for example, the matrix parameter of the eigenmatrix)and a status (for example, a volume of currently buffered data) of thedata buffer unit 403, the data scanning controller 404 and the addressgenerator 405 to perform data expansion on the to-be-expanded datamatrix. The data expansion controller 401 may alternatively control,based on a status (for example, a remaining available capacity) of thedata buffer unit 403, the DDR memory data reading controller 402 to reada preset number of pieces of unexpanded data in the image data matrix.

The DDR memory data reading controller 402 is configured to controlledby the data expansion controller 401 to read corresponding data in theimage data matrix, to obtain a to-be-expanded data matrix, read a presetnumber of pieces of unexpanded data in the image data matrix, update theto-be-expanded data matrix based on the unexpanded data, and write theread-out data to the data buffer unit 403.

The data buffer unit 403 is configured to buffer the data read by theDDR memory data reading controller 402, and output expanded data to thePE.

The data scanning controller 404 and the address generator 405 areconfigured to be controlled by the data expansion controller 401 toperform data expansion on the to-be-expanded data matrix.

The PE 406 is configured to perform a multiplication operation on theexpanded data and the eigenmatrix, to implement a convolution operation.

In one embodiment, the data processing apparatus for a CNN may bespecifically integrated in a co-processor device such as a CPU, an FPGA,an ASIC, or a GPU.

The embodiments of the present disclosure further provide a dataprocessing apparatus for a CNN, including one or more processors and astorage medium. The processor includes a co-processor device such as aCPU, an FPGA, an ASIC, or a GPU, and the storage medium may be anon-volatile computer-readable storage medium, configured to store oneor more computer-readable instructions. The one or morecomputer-readable instructions include an obtaining unit, a readingunit, a data expansion unit, and an updating unit. In anotherembodiment, the one or more computer-readable instructions furtherinclude a storage unit. The processor is configured to read the one ormore computer-readable instructions stored in the storage medium, toimplement steps of the data processing method for a CNN and functions ofunits of the data processing apparatus for a CNN in the foregoingembodiments.

Accordingly, in one embodiment of the present disclosure, the obtainingunit 301 obtains a matrix parameter of an eigenmatrix; then, the readingunit 302 reads corresponding data in an image data matrix from based onthe matrix parameter, to obtain a to-be-expanded data matrix; the dataexpansion unit 303 performs data expansion on the to-be-expanded datamatrix based on the matrix parameter, to obtain expanded data; and theupdating unit 304 reads a preset number of pieces of unexpanded data inthe image data matrix, updates, based on the unexpanded data, theto-be-expanded data matrix, and returns to the step of performingexpansion on the to-be-expanded data matrix based on the matrixparameter. In this solution, during a convolution process, the read-outimage data can be multiplexed to implement data expansion, so as toavoid repeatedly reading of some pieces of data, and reduce requirementsof CNN data expansion for data bandwidth or storage space. Therefore, adata processing capability and data expansion efficiency of a processingsystem for a CNN can be improved.

A person of ordinary skill in the art may understand that all or some ofthe steps of the methods in the foregoing embodiments may be implementedby a program instructing relevant hardware. The program may be stored ina computer readable storage medium. The storage medium may include: aread-only memory (ROM), a random access memory (RAM), a magnetic disk,an optical disc, or the like.

A data processing method and apparatus for a CNN according to theembodiments of the present disclosure are described in detail above. Theprinciple and implementations of the present disclosure are describedherein by using specific examples. The descriptions of the embodimentsof the present disclosure are merely used for helping understand themethod and core ideas of the present disclosure. In addition, a personskilled in the art can make variations to the present disclosure interms of the specific implementations and application scopes accordingto the ideas of the present disclosure. Therefore, the content of thespecification shall not be construed as a limit to the presentdisclosure.

What is claimed is:
 1. A data processing method for a convolutionalneural network (CNN), implemented by a computing device, comprising:(a). obtaining a matrix parameter of an eigenmatrix; (b). readingcorresponding data in an image data matrix from a first buffer spacebased on the matrix parameter, to obtain a next to-be-expanded datamatrix, and sending and storing the to-be-expanded data matrix to asecond preset buffer space; (c). reading the to-be-expanded data matrixfrom the second preset buffer space, and performing data expansion onthe to-be-expanded data matrix based on the matrix parameter, to obtainexpanded data; (d). reading a preset number of pieces of unexpanded datain the image data matrix from the first buffer space, sending andstoring the unexpanded data to the second preset buffer space, andupdating, based on the unexpanded data, the to-be-expanded data matrixstored in the second preset buffer space; and (e). repeating (c) and (d)until all data in the image data matrix is completely read out on theto-be-expanded data matrix based on the matrix parameter.
 2. The dataprocessing method according to claim 1, wherein the performing dataexpansion on the to-be-expanded data matrix based on the matrixparameter comprises: performing data expansion on the to-be-expandeddata matrix based on the matrix parameter and a storage address of dataof the to-be-expanded data matrix in the second preset buffer space. 3.The data processing method according to claim 2, wherein the performingdata expansion on the to-be-expanded data matrix based on the matrixparameter and a storage address of data of the to-be-expanded datamatrix in the second preset buffer space comprises: determining asliding window based on the matrix parameter; sliding the sliding windowon the to-be-expanded data matrix based on a preset sliding directionand a preset sliding step length; obtaining the storage address of datain the sliding window in the second preset buffer space after each timeof sliding; and reading the corresponding data from the second presetbuffer space based on the storage address of data, to complete the dataexpansion.
 4. The data processing method according to claim 2, whereinthe reading a preset number of pieces of unexpanded data in the imagedata matrix from the first buffer space through the first bus comprises:reading the preset number of pieces of unexpanded data in the image datamatrix from the first buffer space based on a first predetermined datavolume through the first bus.
 5. The data processing method according toclaim 4, wherein the reading the preset number of pieces of unexpandeddata in the image data matrix from the first buffer space based on afirst predetermined data volume through the first bus comprises:obtaining a remaining available capacity of the second preset bufferspace; and when the remaining available capacity of the second presetbuffer space is greater than or equal to the first predetermined datavolume, reading the preset number of pieces of unexpanded data in theimage data matrix from the first buffer space based on the firstpredetermined data volume through the first bus.
 6. The data processingmethod according to claim 4, wherein the updating, based on theunexpanded data, the to-be-expanded data matrix stored in the secondpreset buffer space comprises: reading the unexpanded data from thesecond preset buffer space through the second bus, and selecting apreset number of pieces of target data from the unexpanded data; andupdating, based on the target data, the to-be-expanded data matrixstored in the second preset buffer space.
 7. The data processing methodaccording to claim 5, wherein the performing data expansion on theto-be-expanded data matrix based on the matrix parameter and a storageaddress of data of the to-be-expanded data matrix in the second presetbuffer space comprises: obtaining a current buffer data volume of thesecond preset buffer space; and when the buffer data volume is greaterthan or equal to the second predetermined data volume, performing dataexpansion on the to-be-expanded data matrix based on the matrixparameter and the storage address of the data of the to-be-expanded datamatrix in the second preset buffer space.
 8. A data processing system,comprising: a central processing unit (CPU) configured to process dataassociated with a convolutional neural network (CNN); and a co-processorconfigured to performing operations of the CNN, including: (a).obtaining a matrix parameter of an eigenmatrix; (b). readingcorresponding data in an image data matrix from a first buffer spacebased on the matrix parameter, to obtain a next to-be-expanded datamatrix, and sending and storing the to-be-expanded data matrix to asecond preset buffer space; (c). reading the to-be-expanded data matrixfrom the second preset buffer space, and performing data expansion onthe to-be-expanded data matrix based on the matrix parameter, to obtainexpanded data; (d). reading a preset number of pieces of unexpanded datain the image data matrix from the first buffer space, sending andstoring the unexpanded data to the second preset buffer space, andupdating, based on the unexpanded data, the to-be-expanded data matrixstored in the second preset buffer space; and (e). repeating (c) and (d)until all data in the image data matrix is completely read out on theto-be-expanded data matrix based on the matrix parameter.
 9. The dataprocessing system according to claim 8, wherein the performing dataexpansion on the to-be-expanded data matrix based on the matrixparameter comprises: performing data expansion on the to-be-expandeddata matrix based on the matrix parameter and a storage address of dataof the to-be-expanded data matrix in the second preset buffer space. 10.The data processing system according to claim 9, wherein the performingdata expansion on the to-be-expanded data matrix based on the matrixparameter and a storage address of data of the to-be-expanded datamatrix in the second preset buffer space comprises: determining asliding window based on the matrix parameter; sliding the sliding windowon the to-be-expanded data matrix based on a preset sliding directionand a preset sliding step length; obtaining the storage address of datain the sliding window in the second preset buffer space after each timeof sliding; and reading the corresponding data from the second presetbuffer space based on the storage address of data, to complete the dataexpansion.
 11. The data processing system according to claim 9, whereinthe reading a preset number of pieces of unexpanded data in the imagedata matrix from the first buffer space comprises: reading the presetnumber of pieces of unexpanded data in the image data matrix from thefirst buffer space based on a first predetermined data volume through afirst bus.
 12. The data processing system according to claim 11, whereinthe reading the preset number of pieces of unexpanded data in the imagedata matrix from the first buffer space based on a first predetermineddata volume through the first bus comprises: obtaining a remainingavailable capacity of the second preset buffer space; and when theremaining available capacity of the second preset buffer space isgreater than or equal to the first predetermined data volume, readingthe preset number of pieces of unexpanded data in the image data matrixfrom the first buffer space based on the first predetermined data volumethrough the first bus.
 13. The data processing system according to claim11, wherein the updating, based on the unexpanded data, theto-be-expanded data matrix stored in the second preset buffer spacecomprises: reading the unexpanded data from the second preset bufferspace through a second bus, and selecting a preset number of pieces oftarget data from the unexpanded data; and updating, based on the targetdata, the to-be-expanded data matrix stored in the second preset bufferspace.
 14. The data processing system according to claim 12, wherein theperforming data expansion on the to-be-expanded data matrix based on thematrix parameter and a storage address of data of the to-be-expandeddata matrix in the second preset buffer space comprises: obtaining acurrent buffer data volume of the second preset buffer space; and whenthe buffer data volume is greater than or equal to the secondpredetermined data volume, performing data expansion on theto-be-expanded data matrix based on the matrix parameter and the storageaddress of the data of the to-be-expanded data matrix in the secondpreset buffer space.
 15. A non-transitory computer-readable storagemedium storing computer program instructions executable by at least oneprocessor to perform: (a). obtaining a matrix parameter of aneigenmatrix; (b). reading corresponding data in an image data matrixfrom a first buffer space based on the matrix parameter, to obtain anext to-be-expanded data matrix, and sending and storing theto-be-expanded data matrix to a second preset buffer space; (c). readingthe to-be-expanded data matrix from the second preset buffer space, andperforming data expansion on the to-be-expanded data matrix based on thematrix parameter, to obtain expanded data; (d). reading a preset numberof pieces of unexpanded data in the image data matrix from the firstbuffer space, sending and storing the unexpanded data to the secondpreset buffer space, and updating, based on the unexpanded data, theto-be-expanded data matrix stored in the second preset buffer space; and(e). repeating (c) and (d) until all data in the image data matrix iscompletely read out on the to-be-expanded data matrix based on thematrix parameter.
 16. The non-transitory computer-readable storagemedium according to claim 15, wherein the performing data expansion onthe to-be-expanded data matrix based on the matrix parameter comprises:performing data expansion on the to-be-expanded data matrix based on thematrix parameter and a storage address of data of the to-be-expandeddata matrix in the second preset buffer space.
 17. The non-transitorycomputer-readable storage medium according to claim 16, wherein theperforming data expansion on the to-be-expanded data matrix based on thematrix parameter and a storage address of data of the to-be-expandeddata matrix in the second preset buffer space comprises: determining asliding window based on the matrix parameter; sliding the sliding windowon the to-be-expanded data matrix based on a preset sliding directionand a preset sliding step length; obtaining the storage address of datain the sliding window in the second preset buffer space after each timeof sliding; and reading the corresponding data from the second presetbuffer space based on the storage address of data, to complete the dataexpansion.
 18. The non-transitory computer-readable storage mediumaccording to claim 16, wherein the reading a preset number of pieces ofunexpanded data in the image data matrix from the first buffer spacecomprises: reading the preset number of pieces of unexpanded data in theimage data matrix from the first buffer space based on a firstpredetermined data volume through a first bus.
 19. The non-transitorycomputer-readable storage medium according to claim 18, wherein thereading the preset number of pieces of unexpanded data in the image datamatrix from the first buffer space based on a first predetermined datavolume through the first bus comprises: obtaining a remaining availablecapacity of the second preset buffer space; and when the remainingavailable capacity of the second preset buffer space is greater than orequal to the first predetermined data volume, reading the preset numberof pieces of unexpanded data in the image data matrix from the firstbuffer space based on the first predetermined data volume through thefirst bus.
 20. The non-transitory computer-readable storage mediumaccording to claim 18, wherein the updating, based on the unexpandeddata, the to-be-expanded data matrix stored in the second preset bufferspace comprises: reading the unexpanded data from the second presetbuffer space through a second bus, and selecting a preset number ofpieces of target data from the unexpanded data; and updating, based onthe target data, the to-be-expanded data matrix stored in the secondpreset buffer space.